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Executive Summary 

There is a strong case for electronic journals for research in Design 

From a review of practice and thinking in other fields, it appears that there are 
suitable exemplars, tools and methods available which makes such journals viable. 

We propose a two-part approach consisting of a single archive of "publications" 
providing an environment for several refereed research journals and other related 
ventures which could benefit research, teaching and professional practice. 

We propose an approach to progressive refereeing, based on recent thinking and 
developing practice in electronic research publishing, which takes advantage of the 
online environment to support work in development and provide a forum for scholarly 
discourse. 

We propose an outline of the quality requirements and structures needed to ensure 
quality of publication. 

Sheffield Hallam University is setting up and trialling the networked software and 
database needed to develop these proposals, during the next year we would like to 
form a partnership with several institutions to bring the project to reality. 



Background 

This proposal arises from a number of circumstances: 

1 . There is a growing body of research in which the processes and outcomes of 
design practice form a significant part of the investigation or the material 
investigated. In many cases it has been problematic for researchers to provide a 
full permanent record of their work because of the limitations of traditional printed 
media. This is not a unique problem for Design but it is more central and 
pressing for us than for most other disciplines. 

2. Research in Design is served by a limited number of journals and the relatively 
small academic audience makes it unattractive for publishers. At present there 
are three established international broad-based research journals in the English 
language: Design Studies, Tiie Design Journal and Design Issues and all follow 
the general format for a conventional research journal with written papers and 
monochrome images. An attempt to publish a "designerly" research journal (co- 
design) in the mid 1990's failed, arguably because of the clash between 
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production values and production costs, but even tliat did not attempt to address 
tine problem of providing rich visual material. There have been a number of 
recent initiatives to introduce digital journals but, unlike paper journals, they do 
not fit into an established infrastructure. 

The development of electronic publishing on the World Wide Web has created a 
number of useful resources, a body of developing theories and practices for 
electronic journals, and some novel principles for the organisation of peer- 
reviewed research journals and associated archives. We believe that there are 
sufficient ideas, resources and experience available to allow us to construct an 
infrastructure and start to develop an archive together with journals which both 
grow from and depend on it. 



Why an Archive AND Journals 

So far, journals have been the basis for the research archive. If past copies of a 
journal are maintained by libraries then the work is preserved for future scholars. 
Recently some journal publishers have created digital archives which should be more 
accessible than paper but, since most publishers have allowed only their existing 
subscribers to use it, the benefits have not been felt by the wider community. 

Expensive journal subscriptions are an increasing burden for libraries, especially in 
poor countries. Researchers want the widest possible audience to see their work so 
there has been a growth of research archives independent of journals. The pioneer, 
since 1991, is the Los Alamos Physics Archive (http://xxx.lanl.gov) where authors 
can place research papers in an archive that is open to anybody who has internet 
access. The generic description of such an archive is "pre-print" indicating that the 
author has placed the work in the archive before submitting it to a refereed journal. 

This has caused a crisis for physics journals who are now being asked to consider 
publication of papers which are already in the public domain, but the success of the 
Los Alamos project has been such that many journals have accepted the situation 
and we now have a two-track system where the journal validates the research output 
but is no longer the usual, or most up to date, means of access to current research 
papers. 

Recently, this movement has been underpinned by the Budapest Open Access 
Initiative (BOAI 2002), sponsored by George Soros, which is developing data 
standards to ensure that such archives will all form part of a universal network of 
scholarly literature. A copy of the BOAI is included with this information pack. 

The next step has been the development of online journals that are freely available 
on the web. At the moment these are seen as being less significant than paper 
journals but there is enough development going on to indicate that rigorous e- 
journals should gain status and could supplant paper and there is evidence that 
online papers attract a greater number of citations than those on paper alone. The 
significant issue is that they remove all the costs that publishers traditionally met, 
there are still the editing/refereeing costs but those were always met by academics 
and their employers anyway so they are no worse off than before. 

One of the significant costs and problems of an e-journal is maintaining the computer 
infrastructure and ensuring the continuing integrity of the archive. Since these costs 
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and problems are being met by archive projects tinere is an increasing interest in tine 
idea tinat journals should not get involved in the technical problems but rely on the 
archives to look after the material, with virtual journals concentrating on refereeing. 
One proposal (Edmonds 2000) implies that the title "Journal" may become 
redundant, describing a new organisation, a "Review Board" which will provide the 
refereeing function for a specialist area of research, and suggesting that an archive 
may be used/served by many review boards. 

This structure would be ideal for a developing area such as ours since, if we can set 
up a sufficiently flexible and well-structured archive, we can then start to promote the 
development of journals in several areas of Design and the technical entry barrier to 
a new virtual journal will be minimal. There are, of course, quality dangers here but 
those are dealt with below. Another benefit is that the same infrastructure might be 
used for a number of other initiatives, for example allowing archiving and 
"publication" of work by students and practitioners and how this could be of benefit 
will be seen in the discussion of refereeing/review processes below. 



Refereeing and Editing Process 

(NB, Frequently in this discussion we have referred to "papers" and "authors". This 
just reflects the examples and discussions that we have looked at, mainly in the 
sciences and humanities. Our project would be concerned with diverse forms of 
output but the underlying principles are the same) 

The practice of pre-publishing unrefereed papers/work in progress in archives has 
led to the idea that publishing could be a more progressive and transparent process 
than in the old journal system. Andrew Odiyzko (1994) proposed that papers might 
be posted on the web, subjected to informal peer review and revision, leading to a 
formal review by recognised referees. One interesting aspect of his proposal was 
that you could improve and update your paper but you had to leave the earlier 
versions in place as well as the new one. This system does make it impossible to 
carry out blind refereeing but one should question how "blind" refereeing is, 
especially in a small field where people know each other. Certainly, as referees, we 
find that we can usually tell who has written a paper or at the very least where they 
are and where they fit in the picture of design research. The Biomed Central Project 
(www.biomedcentral.com) is the largest open access publishing project to date and 
many of its medical journals use Odiyzko's approach to open refereeing with the full 
process open to inspection by anybody who cares to look. 

Bruce Edmonds (2000) has taken this further by proposing that the refereeing 
process should not be, as at present, a gatekeeping task (is this paper in or out?) but 
instead the referees' ratings (against a range of criteria as used in most refereeing 
processes) and comments for each paper should be published in the archive with the 
paper, allowing readers to see the strengths and weaknesses of each paper and 
choose their own criteria for what they wish to read. For example if there are no 
highly rated papers on a topic, you may wish to see anything at all published on it 
since there is always something to be learned, especially if you are able to see what 
experts have thought about it. A copy of Bruce Edmonds' paper is included with this 
information pack. 

We have discussed this with two of our newer researchers who are very attracted to 
the idea that they could publish their work experimentally, and hope to get some 



An Electronic Archive and Journal/s for research in Design 

Chris Rust, Sheffield Hallam University, 



feedback, rather than struggle with the problem of submitting work for formal 
refereeing and be unsure why or how far they had fallen short of the criteria. We 
would certainly want to see this opportunity used by postgraduate students who need 
to take their first steps into the public domain, whether for research or creative work. 

One might be concerned about whether people would want to give up time to provide 
informal feedback. We suggest that there are three things that need to be done to 
ensure that other researchers and designers want to get involved: 

1 . Firstly one has to establish guidance for "authors" on quality thresholds, and 
perhaps require some local review within the author's own institution, before a 
publication is posted. However these should be seen as opportunities to reflect 
on whether the publication is ready to post rather than gatekeeping. In the end 
the author should be responsible for their own publication and feel properly 
aware of the dangers of exposing poor thinking or poor communication to their 
peers, although there would be some merit in the idea of requiring a sponsor - 
another researcher who is willing to affirm that they believe that this publication is 
ready for posting. 

2. Secondly the notification and search system/s will be crucial. The audience 
should be able to specify their research and professional interests and the author 
must provide enough information (keywords and abstract) to allow the archive 
system to distribute notifications to the right people and to allow the audience 
members to decide whether to look at the full publication. However a good 
notification system should be a considerable bonus for researchers and 
professionals who want to be kept up to date. In design there is a big gap 
between the research and professional communities and this might be a 
excellent way to narrow that gap. 

3. The feedback/discussion must have some impact and create involvement. One 
journal which demonstrates good practice is Pediatrics (http://www.pediatrics.org) 
which has a section called "post-publication peer review" {P^R). This offers an 
insight into how the first stages of the Odiyzko system might work - alongside the 
paper you can read all the comments by peers plus the author's responses in 
return. The papers and review comments are accessible to anybody but only 
registered subscribers can comment, which appears to restrict the comment to 
knowledgeable people although there is the occasional personal rant which is not 
particularly relevant or helpful to the author. 

To summarise, we suggest that any project sets out to develop a progressive 
approach which encourages the public development of work in progress, rather than 
a traditional expert gatekeeper approach. Having looked at examples in action we 
are reasonably confident that the software available {eprints) is capable of 
supporting such a system and that it may be suitable for other purposes as well as 
research refereeing. 

A secondary benefit which could flow from a progressive review approach is that it 
will encourage scholarly debate which focuses on actual research. There are many 
forums for debate on the web but they suffer from lack of discipline and lack of clarity 
about the purposes of a debate, so there is an inevitable drift and trivialisation 
leaving no issues resolved. With rules of engagement and some editorial control (to 
remind people of the need to focus on the publication in hand) a more constructive 
form of debate may emerge, more in the spirit of a learned society. 
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Quality Issues 

Refereeing is at the heart of a research journal. In the past this has been in the 
hands of an editor, who will have a close relationship with the publisher. There is a 
great deal of potential for inertia here, since it makes life easier for the publisher to 
stick with the devil they know, so the editor has a great deal of power and patronage, 
inherited from the publisher who controls the limited means of dissemination. 

We would not suggest that any of the established journals in design have suffered 
problems in this way, but it is arguable that a more open and quality-driven system 
might be possible once the disproportionate influence of the publisher has been 
removed. 

One problem, however, is that, if there are no infrastructure barriers, anybody could 
start a journal and there would be rapid increase in vanity publishing leading to 
confusion and loss of faith in the quality of online journals. 

The responsibility for avoiding this danger lies first with those who create the 
infrastructure and subsequently with the academic community that it serves, 
particularly those who undertake to publish journals within the system. When 
creating the infrastructure it is necessary to establish some rules for participation and 
to take a lead in setting up the first journals which must operate to the highest 
standards. Once there is a small community of journals they must take a lead in 
advising, and ensuring the quality of, any new journal which is introduced. 

A practical example of this can be found in the Association of Peer-Reviewed 
Electronic Journals in Religion (http://rosetta.atla-certr.org/apejr/apejr.html) which 
sets criteria for membership based on the policies of the journals, the main ones 
being the standing and pertinence of the editorial board, quality of peer review 
process and a commitment to publishing for open access on the web. 

APEJR has relatively few teeth since it is only a membership organisation, although 
it does provide the vital validation of publication quality. In our project, the existing 
journals will actually hold the keys to the infrastructure so they will have a greater 
control over quality of publication but a commensurate responsibility to maintain their 
own quality and operate fair and reasonable policies. One central issue is to ensure 
good standards without excluding research which is unconventional or challenging to 
the existing "editors' club". 

So the quality challenge is to: 

1 . Create a technical infrastructure which facilitates good practice in editing and 
peer review. 

2. Establish robust criteria for the organisation and operation of journals. 

3. Set up and support the initial journals, ensuring that they operate to high 
standards and gain international standing. 
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4. Create an organisational framework for tine journals to operate as a self 
regulating association, which both sets standards for new journals and ensures 
the continuing quality of all members. 

5. As in all aspects of a project like this - to make policies and processes very clear 
and explicit to the audience, who have the power to accept or reject e-journals as 
being relevant and of good standing. 



Ownership? 

Our proposal is for the infrastructure and the first journals to be set up by a 
partnership between SHU and other universities with good research standing in 
Design. However the medium term aim must be for ownership of the journals to pass 
to the editors and their association, and the Universities to become providers of the 
technical infrastructure, with the journals (and others who create activities based on 
the archive) as clients. It would be unwise for a university to have a stake in a 
journal if the journal is to be seen as independent. 

Who pays for the infrastructure is open to question. Initially we propose that SHU 
and any partners fund the development, but funding should be sought from AHRB or 
from J ISC who have supported some similar projects. 

We would not favour charging client journals as they have no direct income and 
provide their services as part of their scholarly activity, but it may be worthwhile to 
consider a subscription or filing charge to organisations who wish their staff to post 
publications to the archive. This approach has been proposed by Harnad (2001 , 
4.2.viii) who believes that it would be good value for institutions that will save 
increasingly on the cost of journal subscriptions as electronic archives develop and it 
is the commercial; basis for Biomed Central which appears to be paying its way and 
offering good value. There is also scope for a refereeing charge by journals to 
support their costs. 

We need to consult with scholarly associations in Design, such as European 
Academy of Design, Design Research Society, and the Asian Design Societies. 

Similarly, we would wish to consult with the editors of existing journals, and later on 
invite them to consider moving to electronic publication. However the wider debate 
on e-journals indicates that editors may be hostile and may argue strongly to 
preserve the status quo so we would not wish to expend too much energy revisiting 
arguments which have been rehearsed at length in other fields. The aim is to set up 
and develop electronic journals in response to a need which is not being met, not to 
find reasons to do nothing. The project may not achieve all its aims but, even without 
formal journals, an online archive is of great value, as demonstrated by the Los 
Alamos project. 
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Content, Style and Quality of Communication 

In an archive concerned with Design there is a natural concern with quality of 
communication and a wish to maintain professional standards, especially as it should 
speak to professional as well as academic audiences. In addition we anticipate that 
material will be submitted in a wide range of formats, reflecting the great variety of 
contexts and media that the research will address. 

When dealing with creative people employing diverse media, it would be 
inappropriate and counterproductive to enforce rigid styles, as used in conventional 
research journals. However the danger that people will submit work which is 
incoherent, badly organised or inappropriate in its presentation is ever present and 
there will be a need for guidance at several levels, ranging from discussion of the 
design and communication problems of this type of research through to specific 
exemplars to illustrate the discussion and help inexperienced authors. 

Beyond that there is the more profound issue of how one creates a research 
narrative which makes appropriate and full use of non-text material and this will be 
an important concern in the early years of the archive. We have made a start in 
developing and disseminating ideas about this (eg Rust & Wilson 2001 , Rust & 
Robertson 2003) and we see this project as a vehicle for promoting debate and 
experiment. In the early period of establishing the archive we should seek to foster 
and develop examples of practice which will inform the guidance provided to authors 
as well as the wider practices of research in Art and Design. 



Technical Issues 

The eprints software (http://www.eprints.org/) developed by Southampton University 
with funding from J ISC appears to provide an appropriate environment for this 
project. It conforms to the Budapest standards for research publication archives and 
has been taken up by a number of centres. SHU has installed an eprints server for 
this project and at the moment we are setting up the basic configuration and 
interface to allow some trials to take place. Following the Barcelona workshop we will 
be inviting participation in the trials to develop the usability and usefulness of the 
archive. 

We need to have a clear set of requirements for the system, on the next page there 
is a diagram giving some of the features that we might look for. 
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Some of the requirements for a research publication archive 



Easy for authors to post to and 
update 

Accepts multiple formats, eg: 



Allows Registration 



• Supports continuing review and 
exchanges between authors 
and reviewers 

• Editorial Control 

• Notification to readers with 
abstracts, based on readers, 
interests 



Refereeing process 



Citation Index (there is no 
comprehensive citation index 
for design research) 



Acrobat 

Word 

PowerPoint 

Quicktime 

HTML 

Flash 

Director 

Authors/Reviewers (CV and 
confirmation of status) 
Readers (specify Interests) 

Public, for "pre-press" 

Private, for Editors and Referees 

Allows version updates 

To limit inappropriate behaviour 

New archive entries (email) 
New Refereed Papers (email) 

(on website) 

Mechanisms to ease Editors' role 
Attachment of ratings to papers? 
Subsequent comment by readers? 

Approach which integrates archive- 
based journals with abstracts and 
bibliographies from papers published 
elsewhere 



• Author CV's 



Generates comprehensive listing and 
links to all publications and other 
contributions by each "author" as well as 
other information provided by them 



in addition there are technical concerns about longevity of digital archives William 
Arms (1999) gives an overview of the main problems. Immediate problems are 
ensuring that the archive is mirrored on several sites and that there are strategies for 
regular migration of the data to avoid deterioration of the media. In the longer term it 
is important to have strategies for ensuring the continuance of the archive if the 
original sponsor (eg SHU) withdraws or ceases to exist. Unlike paper archives which 
will sit on library shelves until somebody decides to throw them away, digital material 
can just disappear from lack of attention. 
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A further problem is the need to ensure that archived material is not tampered with 
after posting since there will be temptations for authors and others to alter history. 
This is already a serious problem for web-based material and one of the main 
arguments for a managed archive rather than links to institutional or author's pages. 



Conclusions 

So, to sum up, we have an opportunity to create something new and useful for 
research in Design. To make progress in the forthcoming year we need to: 



Establish appropriate partnerships 

Establish "Editorial Boards" for the archive and journals, bringing in people of 

good standing from the wider subject communities. 

Develop formal policies for the archive and one or two pilot journals. 

Create the archive, place some trial material in it and test run/develop the full 

range of activities. 

Launch the project to the design community, and be pro-active in encouraging 

colleagues to start to post material and seek refereeing, probably with an 

experimental period when they can pull it back out if they have done something 

embarrassing. 

Recruit existing journal editors and conference organisers to use the archive as 

their publishing infrastructure 
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